Python: dictionary of functions

2009-07-03

On twitter recently I was wondering what the best way to create a dictionary of functions in Python was. On a suggestion of Paul Hankin’s I looked into classes and metaclasses. The most direct way I discovered is to use a class; metaclass not required:

class foo:
  def bar(): pass
  def zon(x): return 1+x
fundict = foo.__dict__

This is direct, but not at all obvious. Hint to people that are mystified already: the __dict__ attribute of the class is a dictionary containing everything defined in the class.

fundict is now a dictionary that contains the two functions bar and zon.

I made this discovery a few days before EuroPython, and was fortunate enough to bump into Bruce Eckel in the hallway at EuroPython just before he gave his metaprogramming talk, so I showed it to him in what I call my “naughty decorator” form:

def asdict(x):
  return x.__dict__

@asdict
class baz:
  def bar(): pass
  def zon(x): return 1+x

(By the way, one of the things I learnt at EuroPython is that the crappy decorator syntax gives Java refugees a warm fuzzy feeling).

Bruce pronounced this “not as naughty as you imply” and “worth showing” (this blog post was half-written before I bumped into Bruce, but he is definitely encouraging me).

It’s worth showing for the following reason:

The functions you get from the class’s __dict__ dictionary are not the same as the methods you get by accessing the attributes of the class. In other words «foo.bar is not foo.__dict__['bar']». I was surprised by this, and so was Bruce.

As well being a little weird, it makes a compact example to show the different between a function, an unbound method, and a bound method.

I hope I don’t need to introduce a function. It’s just a thing you call with some arguments. Where it differs from a method is that a method is regarded as a message sent to an object, and receives that object as its first argument.

In Python, an unbound method is a method that isn’t associated with any particular object; it requires an object as its first argument (and Python loses big here, by requiring the object to be an instance of the class that defined the method). A bound method is a method already associated with an object; that object becomes the method’s first argument when the bound method is invoked with the remaining arguments.

So if we define a class foo (as we did, above), then:

foo.bar is an unbound method;

foo().bar is a bound method (bound to the object we just created by invoking the foo class); and,

foo.__dict__[‘bar’] is a function.

This last fact was a great surprise to me. I had expected it to be an unbound method, and thought that my naughty decorator would have to have some hacky code to dig the function out of the unbound method. But it doesn’t.

Tiny problem: Using the asdict gives a dictionary that contains __module__ and __doc__ keys. Solution: another decorator:

def cleandict(x):
  for k in ('__module__', '__doc__'):
    del x[k]
  return x

@cleandict
@asdict
class baz:
  def bar(): pass
  def zon(x): return 1+x

This entry was posted on 2009-07-03 at 12:34:47 and is filed under python.

4 Responses to “Python: dictionary of functions”

  1. Simon Davy Says:

    2009-07-04 at 09:33:34

    Hello fellow europythoneer :)

    The reason your get different versions of the callable from attribute lookup on the class and key lookup in the __dict__ is to do with the descriptor protocol.

    When you access something via the ‘.’ attribute syntax, some additional steps are performed. After the object (a function in your case) is retrieved via key look up from the dict, some special attributes are looked for on that object: __get__, __set__, and __delete__ (depending on the action being performed). If any of these exist, the descriptor protocol returns the results of calling the appropriate function.

    For methods, they have a __get__ method defined on them (via the class declaration syntax) that converts them to bound methods and inserts the class instance as the first parameter (i.e. self), and returns that function to be called.

    This is also the way that properties are implemented.

    In Raymond Hettinger’s word’s “we control the dot”. I can’t think of another language that allows you to customise attribute lookup in this way. More python awesomeness.

    HTH – googling “python descriptor protocol” will reveal much more info.

    Reply
    • drj11 Says:

      2009-07-06 at 07:11:56

      Common Lisp allows a similar level of customisation. The getter and setter for a slot are just a pair of methods. You can define them how you like.

      Reply
  2. Paddy3118 Says:

    2009-07-07 at 17:31:18 
    >>> def foo(): pass

>>> def bar(): pass

>>> funcdict = dict( (f.func_name, f) for f in [foo, bar] )
>>> funcdict['foo']

>>> funcdict['bar']

>>>

    The above leaves you expecting to create a dict of funcs whereas using class, you expect a claass as the result and the effect of accessing __dict__ of a class is far from ccleaar.

    – Paddy.

    Reply
  3. drj11 Says:

    2009-07-08 at 07:37:25

    Yes. That’s why I used the word “direct” instead of “clear” in my blog post. The clear way is:

    def foo(): pass
def bar(): pass
fundict = dict(foo=foo, bar=bar)

    It mentions the function names three times each. I kind of like the way you (@Paddy3118) use the func_name attribute to avoid that, but even then you still have to mention each function name twice.

    Another way I discovered of creating a dictionary of functions was this:

    def mkit():
  def foo(): pass
  def bar(): pass
  return locals()
fundict = mkit() ; del mkit

    Again, not clear. And using locals() like that feels a bit like the kind of thing that might break at the whim of an implementor.

    Now that this comment is turning this article into Extended Edition, I should note that a dictionary of functions is very straightforward and clear in JavaScript and Lua. That’s because their lambdas are not broken (spelled using the “function” keyword in both languges, as it happens).

    Reply

Leave a comment